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WHAT IS CLAIMED IS: 

An image processing apparatus comprising 
labeling means for extracting frame image data 
from moving image data, segmenting the frame Amage data 
into blocks, and assigning labels in accordance with 
feature amounts obtained in units of bloc] 

sequential label set generation meap^s for 
generating a sequential label set by ari/anging the 
labels assigned by said labeling mean^/ in a 
predetermined block order; 

sequential label set accumulation means for 
accumulating the sequential label- set generated by said 
sequential label set generatijm means in connection 
with the frame image data; 

similarity computat/on means for computing 
similarities between thd generated sequential label set 
and sequential label yBets of a previous frame image 
data group; 

scene chang^ detection means for detecting a 
scene change fr/me in the moving image from a group of 
computed similarities; and 

scene/change storage means for storing 
information of the detected scene change frame in 
connect/on with the frame image data. 

2. /The apparatus according to claim 1, wherein the 
information of the detected scene change frame includes 
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the number of frames or an elapsed time from the 
beginning of the moving image to the detected sqene 
change frame . 

3. The apparatus according to claim 1, wherein the 
5 labels are unique labels which are given to individual 

cells obtained by segmenting a multi-dimensional 
feature amount space into a plurality of cells, and 
said labeling means computes a feature amount for each 
block, and assigns to that block a lalyel given to the 
10 cell to which the computed feature amount belongs. 

4. The apparatus according to c/aim 3, wherein the 
moving image is a color image, tbfe feature amount 
corresponds to a position of a/color element value in 
the multi-dimensional featur/ amount space, and the 

15 labels are unique labels g/ven to individual cells 

obtained by segmenting "Lne multi-dimensional feature 
amount space into a plurality of cells 

5. The apparatus Recording to claim 1, wherein the 
plurality of block^s are obtained by segmenting an image 

20 into a plurality/of vertical and horizontal blocks, and 
the block orde/ used in said sequential label set 
generation mafans is an order in which the plurality of 
blocks are /scanned in a horizontal or vertical 
direct ior 

25 6. Tbfe apparatus according to claim 1, wherein said 
scene /change detection means comprises determination 
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means for determining a scene change when the 
similarity computed by said similarity computation 
means is not more than a predetermined va/ue . 

7. The apparatus according to claim A, wherein said 
5 similarity computation means has a penalty table for 

holding penalty vales in correspondence with pairs of 
label values, acquires penalty values by referring to 
said penalty table using pairs/of label values obtained 
from a sequential label set jof the frame image data of 
10 a scene change frame candidate and sequential label 
sets of the previous frame image data group, and 
computes the similarity on the basis of the acquired 
penalty values, 

8. The apparatu/ according to claim 7, wherein the 
15 labels are unique/ labels which are given to individual 

cells obtained fc/y segmenting a multi-dimensional 
feature amount/space into a plurality of cells, and the 
penalty value/ is a value set based on a distance 
between celys expressed by two. labels, 
20 9. The apparatus according to claim 7, wherein said 

similarity computation means also gives penalty values 
represerycing degrees of similarity of the labels upon 
computing the similarity between the sequential label 
set of the frame image data of the scene change frame 
25 candidate and the sequential label sets of the previous 
frame image data group. 
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10. The apparatus. according to claim 7, wherein saip 
similarity computation means computes the similarit/es 
by DP matching using the penalty values. / 

11. The apparatus according to claim 10, wher/in said 
5 similarity computation means further comprised setting 

means for setting a width of a matching window of DP 
matching to be used. / 

12. The apparatus according to claim 1, wherein the 

ri sequential label set represents a tzwo-dimensional label 

!fl 10 matrix set, and / 

111 / 

= =7h said similarity computat/on means comprises: 

first matching means fjbr corresponding sequential 
^ label sets in units of lines extracted from a label 

]zt matrix of frame image daca of a scene change frame 

15 candidate, and sequential label sets in units of lines 
O extracted from a lab'el matrix of previous frame image 

data by DP matching to obtain a line arrangement of the 
extracted the extracted image data; and 

second matching means for obtaining similarity 
20 between a linflS arrangement of the label matrix of the 
frame image yaata of the scene change frame candidate, 
and the lijxe arrangement ' obtained by said first 
matching *neans by DP matching. 

13. Tbre apparatus according to claim 12, wherein the 
25 sequential label sets in units of lines are each an 
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arrangement corresponding to a horizontal direction of/ 
an image . / 

14. The apparatus according to claim 12, wherein/rhe 
sequential label sets in units of lines is an / 

5 arrangement corresponding to a vertical direcfelon of an 
image . / 

15. The apparatus according to claim 12/ further 
comprising frame determination means toj determining 

□ the frame image data of the scene change frame 

yl 10 candidate to be scene change frame image data, when the 

hj similarity obtained by said second /matching means 

becomes not more than a predetermined value, and 
' similarities obtained by said second matching means as 

JfJ a result of the same processed for previous frame 

% 15 images become not more than/the predetermined value. 

J:f 16. The apparatus according to claim 12, wherein said 

first matching means ha6 a penalty table for holding 
penalty values in correspondence with pairs of labels, 
and refers to said /penalty table upon computing a 
20 distance between/a sequential label set of the frame 

image data of ilhe scene change frame candidate and the 
sequential Isabel set of the previous frame image data 
using DP m/tching. 

17. Ther apparatus according to claim 12, wherein said 
25 second /matching means has an inter-line penalty table 

for holding penalty values in correspondence with pairs 
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of line numbers in the line arrangement, and refers to 
the inter-line penalty table upon computing similarity 
between the line arrangement of the frame image d^ta of 
the scene change frame candidate and the line 
5 arrangement of the previous frame image using/DP 
matching . 

18. The apparatus according to claim 17/ further 
comprising penalty holding means for determining 
penalty values corresponding to pairs/ of lines on the 

10 basis of similarities of sequent ial/ label sets of the 

frame image data of the scene change frame candidate in 
the line direction, and holding/ the determined penalty 
values as the inter-line pena/ty table. 

19. The apparatus according to claim 14, wherein said 
15 first matching means gives/ a penalty and constraint 

upon scaling a sequential label set to be compared when 
similarity between a label sequence of source image 
data and a sequential label set stored in said storage 
means is computed 
20 20. The apparatus according to claim 19, wherein the 
penalty and constraint upon scaling the sequential 
label set to bfe compared are acquired on the basis of a 
theory of DB^ matching. 

21. The Apparatus according to claim 12, further 
25 comprising first matching window setting means for 
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setting a width of a matching window of DP matching/ 
used by said first matching means. / 

22. The apparatus according to claim 12, further 
comprising second matching window setting means; for 

5 setting a width of a matching window of DP marching 
used by said second matching means. / 

23. A scene change detection method comprising the 
steps of: / 

Q extracting frame image data frdm moving image 

in 10 data, segmenting the frame image d^ta into blocks, and 

|Tj assigning labels in accordance w/th feature amounts 

;rj obtained in units of blocks; / 

generating a sequential label set by arranging 
!r! the assigned labels in a predetermined block order; 

^ 15 computing similarities between the generated 

O sequential label set arnd sequential label sets of a 

previous frame imag^ data group; and 

detecting a^scene change frame in the moving 
image from a graaip of computed similarities. 
20 24. The metfrod according to claim 23, wherein 

information /of the detected scene change frame includes 
the numbe/ of frames or an elapsed time from the 
beginning of the moving image to the detected scene 
change/ frame. 

25 25. / The method according to claim 23, wherein the 

lab4ls are unique labels which are given to individual 
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cells obtained by segmenting a multi-dimensional / 
feature amount space into a plurality of cells, and the 
step of assigning the labels includes the step lot 
computing a feature amount for each block, ano 
5 assigning to that block a label given to the cell to 
which the computed feature amount belongs^ 

26. The method according to claim 25-/ wherein the 
moving image is a color image, the feature amount 
corresponds to a position of a color element value in 

10 the multi-dimensional feature amount space, and the 
labels are unique labels given/ to cells obtained by 
segmenting the multi-dimensLonal feature amount space 
into a plurality of cells./ 

27. The method according to claim 23, wherein the 

15 plurality of blocks ares obtained by segmenting an image 
into a plurality of Vertical and horizontal blocks, and 
the block order use6 in the step of generating the 
sequential label yset is an order in which the plurality 
of blocks are scanned in a horizontal or vertical 

20 direction. / 

28. The method according to claim 23, wherein the 
step of detecting a scene change includes the step of 
determining a scene change when the similarity computed 
in the /step of computing similarities is not more than 

25 a predetermined value. 
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29. The method according to claim 23, wherein the/ 
step of computing similarities uses a penalty tabLe for 
holding penalty vales in correspondence with paii/s of 
label values, and includes the step of acquiring 
5 penalty values by referring to the penalty tabfle using 
pairs of label values obtained from a label /sequence of 
the frame image data of a scene change frame candidate 
and label sequences of the previous frame image data 
O group, and computing the similarities pn the basis of 

m 10 the acquired penalty values. / 

y 30. The method according to claim/29, wherein the 

%j labels are unique labels which ar/ given to individual 

cells obtained by segmenting a r/ulti-dimensional 
j-Tj feature amount space into a plurality of cells, and the 

15 penalty value is a value selz based on a distance 

/ 

:;f between cells expressed kw two labels. 

31. The method according to claim 29, wherein the 
step of computing similarities includes the step of 
also giving penalty values representing degrees of 

20 similarity of labels upon computing the similarity 
between the sequential label set of the frame image 
data of the sc/ne change frame candidate and the label 
sequence of tme previous frame image data. 

32. The method according to claim 29, wherein the 
25 step of oomputing similarities includes the step of 
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computing the similarities by DP matching using tpe 
penalty values. 

33. The method according to claim 32, whereirf the 
step of computing similarities includes the s^ep of 

5 setting a width of a matching window of DP patching to 
be used. 

34. The method according to claim 23, /wherein the 
sequential label set represents a twg/dimensional label 
matrix, and 

10 the step of computing similarities includes: 

the first matching step oT corresponding 
sequential label sets in unit's of lines extracted from 
a label matrix of frame im^ge data of a scene change 
frame candidate, and sequential label sets in units of 
15 lines extracted from a /abel matrix of previous frame 
image data by DP matching to obtain a line arrangement 
.of the extracted the/ extracted image data; and 

the second m/tching step of obtaining similarity 
between a line arrangement of the label matrix of the 
20 frame image datA of the scene change frame candidate, 
and the obtained line arrangement by DP matching, 

35. The method according to claim 34, wherein the 
sequential /label sets in units of lines are each an 
arrangeme/it corresponding to a horizontal direction of 

25 an imagof. 
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36. The method according to claim 34, wherein yhe 
sequential label sets in units of lines are each an 
arrangement corresponding to a vertical direct/ion of an 
image . 

5 37. The method according to claim 34, wherein the 

frame image data of the scene change frame candidate is 
determined to be scene change frame image data, when 
the similarity obtained in the secoixd matching step 
becomes not more than a predetermined value, and 
10 similarities obtained in the seo6nd matching step as a 
result of the same processes fior previous frame images 
become not more than the predetermined value. 

38. The method according^ to claim 34, wherein the 
first matching step use/ a penalty table for holding 

15 penalty values in correspondence with pairs of labels, 
and includes the ste^p of referring to the penalty table 
upon computing a distance between a sequential label 
set of the framer image data of the scene change frame 
candidate and/the sequential label set of the previous 

20 frame image /data using DP matching. 

39. The lriethod according to claim 34, wherein the 
second m/tching step uses an inter-line penalty table 
for holding penalty values in correspondence with pairs 
of liyfe numbers in the line arrangement, and includes 

25 the yfetep of referring to the inter-line penalty table 
uptfn computing similarity between the line arrangement 
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of the frame image data of the scene change frar 
candidate and the line arrangement of the previous 
frame image using DP matching, 

40. The method according to claim 39, wherein penalty 
5 values corresponding to pairs of lines are/determined 
on the basis of similarities of sequential label sets 
of the frame image data of the scene change frame 
candidate in the line direction, and/ the determined 
penalty values are held as the in^/er-line penalty table. 

10 41. The method according to cl^im 36, wherein the 
first matching step includes t/ne step of giving a 
penalty and constraint upon /Scaling a sequential label 
set to be compared when similarity between a sequential 
label set of source imagf'e data and a sequential label 

15 set stored in storage yfneans is computed, 

42. The method acoording to claim 41, wherein the 
penalty and constraint upon scaling the sequential 
label set to be Compared are acquired on the basis of a 
theory of DP matching, 

20 43. The method according to claim 34, wherein a width 
of a matching window of DP matching used in the first 
matching step is set. 

44. The method according to claim 34, wherein a width 
of a retching window of DP matching used in the second 
25 matofiing step is set 
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45. A storage medium for storing a control prograr 
for making a computer execute scene change detection, 
said control program including: 

the step of extracting frame image data frjbm 
5 moving image data, segmenting the frame image jdata into 
blocks, and assigning labels in accordance w/th feature 
amounts acquired in units of blocks; 

the step of generating a sequential label set by 
arranging the assigned labels in a predetermined block 
10 order; 

the step of computing similarities between the 
generated sequential label set ary3 sequential label 
sets of a previous frame image yflata group; and 

the step of detecting e/ scene change frame in the 
15 moving image from a computafl similarity group, 

46. The medium according to claim 45, wherein the 
labels are unique label/ which are given to individual 
cells obtained by segntfenting a multi-dimensional 
feature amount spac^ into a plurality of cells, and the 

20 step of assigning /the labels includes the step of 
computing a feature amount for each block, and 
assigning to ttoat block a label given to the cell to 
which the computed feature amount belongs. 

47. The inedium according to claim 45, wherein the 
25 step of otomputing similarities includes the step of 
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computing the similarities by DP matching using penalt/y 
values. / 

48. The medium according to claim 45, wherein the 
sequential label set represents a two-dimensiona2: label 

5 matrix, and / 

the step of computing similarities includes: 
the first matching step of corresponding 
sequential label sets in units of lines extracted from 
Q a label matrix of frame image data off a scene change 

m 10 frame candidate, and sequential Label sets in units of 

i'7| lines extracted from a label matrix of previous frame 

7j image data by DP matching to/obtain a line arrangement 

■=s..S / 

of the extracted the extracted image data; and 

the second matching step of obtaining a 

*Z/ 15 similarity between a iiine arrangement of the label 

± / 

O matrix of the frame' image data of the scene change 

frame candidate, /and the obtained line arrangement by 
DP matching. / 

49. The medium according to claim 48, wherein the 

20 frame imacpe data of the scene change frame candidate is 
determined to be scene change frame image data, when 
the similarity obtained in the second matching step . 
becon/es not more than a predetermined value, and 
similarities obtained in the second matching step as a 

25 rersult of the same processes for previous frame images 
become not more than the predetermined value. 
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50. The medium according to claim 48, wherein thd 
first matching step uses a penalty table for holdang 
penalty values in correspondence with pairs of /abels, 
and includes the step of referring to the penalty table 
upon computing a distance between a sequential label 
set of the frame image data of the scene/ change frame 
candidate and the sequential label set? of the previous 
frame image data using DP matching./ 

51. The medium according to clai™ 48, wherein the 
second matching step uses an in/er-line penalty table 
for holding penalty values in/ correspondence with pairs 
of line numbers in the line/arrangement, and includes 
the step of referring to yche inter-line penalty table 
upon computing similari/uy between the line arrangement 
of the frame image da/ca of the scene change frame 
candidate and the Line arrangement of the previous 
frame image using/DP matching. 

52. The medium/ according to claim 45, wherein said 
control program further includes the step of 
determining penalty values corresponding to pairs of 
lines on the basis of similarities of sequential label 
sets of tne frame image data of the scene change frame 
candidaj/e in the line direction, and holding the 
detern^ned penalty values as the inter-line penalty 
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53. The medium according to claim 48, wherein jzhe 
first matching step includes the step of giving a 
penalty and constraint upon scaling a sequential label 
set to be compared when similarity between/a label 

5 sequence of source image data and a sequ^mtial label 
set stored in storage means is compute^ 

54. The medium according to claim/48, wherein the 
first matching step and/or the second matching step 
include/includes the step of setting a width of a 

10 matching window of DP matchi/fig used. 

55. The apparatus accor/ing to claim 11, wherein when 
the apparatus is equipped in a movie, the width of a 
matching window is changed in accordance with a mount 
of shaking detected by a sensor. 

15 56. The method^ according to claim 33, wherein in a 
movie, the wicath of a matching window is changed in 
accordance yWith a mount of shaking detected by a sensor, 
57. The/ medium according to claim 45 wherein in a 
movie, /the width of a matching window is changed in 
20 acco/dance with a mount of shaking detected by a sensor, 
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